Out-of-School Time in Elev8 Community Schools: 
A First Look at Participation and its Unique Contribution 
to Students’ Experiences in School 


Prepared for The Atlantic Philanthropies 

Prepared by Research for Action & McClanahan Associates, Inc. 

July 2013 

Authors 

Wendy S. McClanahan | Jian Gao | Felicia Sanders 




About Research for Action 

Research for Action (RFA) is a Philadelphia-based nonprofit organization. We seek to use research as 
the basis for the improvement of educational opportunities and outcomes for traditionally underserved 
students. Our work is designed to: strengthen public schools and postsecondary institutions; provide 
research-based recommendations to policymakers, practitioners, and the public at the local, state, and 
national levels; and enrich the civic and community dialogue about public education. For more 
information, please visit our website at www.researchforaction.org . 


About McClanahan Associates, Inc. 

At McClanahan Associates, Inc., we are committed to strengthening programs based on flexible, yet 
rigorous, evaluation practices that help organizations and funders achieve their mission of improving 
people’s lives. We work closely with the organizations we partner with - capitalizing on their deep 
knowledge of the local context, the population they serve, and their own program - and generating 
information and recommendations for our clients that are understandable, practical, and useful. Our 
staff is comprised of expert researchers, evaluators and technical assistance providers. As a team, we 
believe that meaningful evaluation efforts respond to each program’s unique needs, align with the 
developmental stage of the program, and produce information that is immediately relevant to 
practitioners. 


Acknowledgements 

This research was made possible with funding from The Atlantic Philanthropies. Research for Action 
and McClanahan Associates, Inc. would like to thank the foundation for its support and its dedication to 
leveling the playing field for vulnerable middle school youth from low income communities. Atlantic 
staff members, Naomi Post and Sandra Trujillo, participated in the development of the report and 
provided a comprehensive review of its final draft. 

This report is the result of work by many organizations and individuals. Thanks go to all of the agencies 
and schools implementing Elev8. In addition to providing valuable services to students, they have 
endured the demands of an intensive research process and reviewed the report for factual 
accuracy. Staff at Research for Action and McClanahan Associates, Inc. were also integral to the 
completion of this report. Meridith Polin worked tirelessly with the sites over the last five years to 
collect data about the initiative and prepared the description of that process for the report. Anurag 
Kumar managed the data, and prepared it for analysis. Emily Lipman pulled the site descriptions 
together. Kate Shaw and Stephanie Levin provided insightful feedback on the final draft. Jenna 
McCready and Jesse Gottschalk copy-edited the report. Alison Murawski and Allison Petrosky designed 
the report and organized its production. 


Table of Contents 

Executive Summary i 

Chapter I: Introduction I 

About this Study 2 

Chapter 2: Community Schools and OST — What We Know 4 

History of Community Schools 4 

Evidence of Community Schools’ Success 4 

Improved Academic Performance 4 

Improved Behavior 5 

Increased Parental Involvement 5 

The Role of OST 5 

Chapter 3: The Elev8 Community Schools Initiative 6 

A Brief History of Elev8 7 

Elev8 Sites: A National Model with Local Customization 7 

Chapter 4: Participation in Elev8 OST 9 

How Many Students Participate in Elev8 OST? 9 

Racial and Ethnic Characteristics of Elev8 OST Participants 10 

Gender 1 2 

Income Status I 3 

Grade Level 1 4 

What Are Students’ Patterns of Participation in Elev8 OST? 14 

Chapter 5: How is Participation in Elev8 OST Related to Students’ Experiences of School? 20 

Comparing the Demographic Characteristics of Participants and Non-Participants 21 

How Is Student Participation in Elev8 OST Activities Related to School Experience and 8 th Graders’ High School 
Transition? 21 

Was Participation in Elev8 OST Related to Attitudes Toward School? 22 

Was Participation in Elev8 OST Related to High School Planning? 23 

Summary 28 

Chapter 6: Final Thoughts 29 

Appendix A: Data Sources 3 1 

Appendix B: Survey Methodology 33 

Appendix C: Survey Measures 35 

Appendix D: Demographic Comparisons of Elev8 Participants with Non-Participants 36 

Appendix E: Regression Analysis 37 


Out-of-School Time in Elev8 Community Schools: 

A First Look at Participation and its Unique Contribution 
to Students’ Experiences in School 


Prepared for The Atlantic Philanthropies 
Prepared by Research for Action & McClanahan Associates, Inc. 


July 2013 


Authors 

Wendy S. McClanahan 
Jian Gao 
Felicia Sanders 


Copyright © 20 1 3 Research for Action & McClanahan Associates, Inc. 




RESEARCH for ACTION 



McClanahan 


Associates 


Out-of-School Time in Elev8 Community Schools: 

A First Look at Participation and its Unique Contribution 
to Students’ Experiences in School 


Executive Summary 

July 2013 


Introduction 

A solid, high-quality education is a key to successful adulthood; however, young people from low- 
income families and communities often face a myriad of educational and academic challenges that put 
them at risk for ongoing disadvantage. Community schools are an approach that has been adopted to 
respond to this educational crisis. By partnering with local agencies, community schools provide 
students and families with access to healthcare services, educational enhancement and recreational 
opportunities, family economic supports such as workforce development and income tax assistance, 
and other programming, like educational advocacy activities, leadership opportunities, child care, and 
others, to increase student outcomes and overall quality of life. 1 

About Elev8 

In 2007, The Atlantic Philanthropies (Atlantic) made an investment in its first five community schools 
in the State of New Mexico. Over the years that followed, Atlantic invested in developing community 
schools in three additional locations —Chicago, Baltimore, and Oakland— totaling about 20 schools 
across all four regions. 

For this initiative, now known as Elev8, Atlantic selected agencies with deep local roots to serve as 
regional leads. Each of these grantees has developed and implemented flexible, full-service community 
school models in up to five schools, focusing on middle-grade students in low-performing schools in 
their region. Elev8 schools employ a team of Out-of-School Time (OST) staff, family advocates, medical 
professionals, a site director, and others, as dictated by their model. 

About this Study 

Since 2008, Atlantic’s evaluation effort had focused on generating information to help create and 
sustain the strongest initiative possible. As a result, the first five years of the evaluation were designed 
to ensure that the model was robustly implemented, and to assist the local sites in utilizing evaluation 
information to strengthen their efforts. However, Atlantic and their grantees are now seeking to 
document how students fare in the program, and to contribute to the literature on community schools. 


1 Bireda, S. (2009). A Look at Community Schools, Center for American Progress. Retrieved November 29, 2012 from 
http://www.americanprogress.org/wp-content/uploads/issues/2009/10/pdf/community_schools.pdf 


With this in mind, the evaluation team set forth to explore what questions we could answer with the 
data we had in hand. As a significant investment is made in OST activities in Elev8 and other 
community school models, this report takes a preliminary look inside the “black box” of community 
schools, thereby providing important information to the field about the relative value of OST within a 
community school model. This study answers the following questions: 

• Who participates in OST in Elev8 schools? 

• What are their patterns of participation? 

• How do Elev8 OST participants compare to students in Elev8 schools who do not participate in 
Elev8 OST (called “non-OST” or “non-participants” in this report)? 

• How is participation in Elev8 OST related to students’ experiences of school? 

To answer these questions, we used data from three main sources: 

• Administrative records containing student demographic data; 

• OST participation data (the “participation data”); and, 

• Self-report data from our annual survey of students in Elev8 schools. 

More details on the data sources and survey administration can be found in the appendices of the full 
report. 


Community Schools and OST — What We Know 

While community schools have existed for almost a century, rigorous evaluation of the impact of this 
educational model is limited, but growing. Research points to three indicators that suggest that 
community schools are a promising approach. 2 They include: 

• Improved academic performance; 

• Improved behavior; and, 

• Increased parental involvement. 

OST activities are a central component of community schools and, while not a lot is known specifically 
about OST in the context of community schools, the literature on the impact of school- and/or 
community-based OST programs is quite robust, though results are mixed. Many programs 
demonstrate increased academic achievement, positive youth development, and reductions in harmful 
behaviors, such as drug use. 3 * 5 However, some studies report more limited benefits to students who 
participate in OST programs. Research suggests that participation rates vary in response to student 
motivation^ the needs of the family; whether that is quality childcare or academic support; and, 
program availability. 3 


2 For a review, see Dryfoos, J.G. (2000). Evaluation of Community Schools: Findings to Date. Coalition for Community Schools. 

3 Weiss, H.B. (2004). Understanding and Measuring Attendance in Out-of-School Time Programs. Issues and Opportunities in Out-of-School- 
Time Evaluation. Harvard Family Research Project. Number 7. 

-* Bodilly, S.J. & Beckett, M.K. (2005). Making Out-of-School-Time Matter. RAND Research Brief Education. Retrieved from 
www.rand.org/publications/MG/MG242 

5 Hynes, K. & Sanders, F. (2010). The changing landscape of afterschool programs. Afterschool Matters (June). 


Participation in Elev8 OST Programming 

Elev8 OST is an essential component of the Elev8 initiative. However, the exact nature of the 
programming varies by region and school. All Elev8 schools provide both recreational and academically 
focused OST programming. 6 Students come into the OST program in a variety of ways. In most cases, 
participation in Elev8 OST activities is open to all students who are interested. In a handful of schools, 
however, students struggling academically are exclusively targeted for Elev8 OST programming which, 
in those schools, is focused on academics. Additionally, some Elev8 schools offer non-Elev8-funded 
OST activities to students, so a student not enrolled in Elev8 OST may still have the opportunity to 
participate in other school-based OST activities that 
are not reflected in our data. 

Who Participates in Elev8 OST Activities? 

The demographic characteristics of participants are 
displayed in Table ES-i. 

• Racial and ethnic characteristics: 

Representative of the regions in which Elev8 
is located. Elev8 OST participants are primarily students of color. 

• Gender: Elev8 OST participants are more likely to be female than male; however, gender 
breakdowns vary by site. 

• Income status: The bulk of students who participate in Elev8 OST, as well as those who attend 
Elev8 schools, are from low-income homes. 7 

• Grade level: Elev8 targets middle-grade students. Overall, the data shows that younger 
students are more likely to participate than older students. This condition holds true in many 
OST programs nationwide. 

Table ES-I: Characteristics of Elev8 OST participants 


Representative of the regions in which 
Elev8 is located, students who participate 
in Elev8 OST are primarily low-income 
students of color. Female students and 
middle schoolers in the younger grades are 
most likely to participate. 


CHARACTERISTIC 

Region A 

Region B 

Region C 

Region D 


N 

% 

N 

% 

N 

% 

N 

% 

RACE/ETHNICITY 

African American 

285 

93.1 

523 

48.7 

109 

37.7 

41 

5.1 

Asian 

0 

0 

3 

0.3 

13 

4.5 

8 

1.0 

Hispanic 

0 

0 

533 

49.7 

148 

51.2 

353 

43.7 

Native American 

0 

0 

3 

0.3 

0 

0 

267 

33.0 

Other 

12 

3.9 

8 

0.8 

19 

6.6 

21 

2.6 

White 

9 

2.9 

3 

0.3 

0 

0 

1 18 

14.6 

GENDER 

Female 

158 

51.6 

563 

52.5 

144 

49.8 

435 

53.8 

Male 

148 

48.4 

510 

47.5 

145 

50.2 

373 

46.2 


6 Only one region provided data that documents the actual types of OST programs that students participate in. That data is provided later in 
this report. 

^ We gauged income status by using free or reduced-price lunch eligibility as a proxy for income (denoted FRL and NON-FRL herein). 








FRL STATUS 


FRL 

0 

0 

989 

92.2 

237 

82.0 

661 

81.8 

NON-FRL 

0 

0 

50 

4.7 

1 0.4 

1 18 

14.6 

Not Sure 

306 

100.0 

34 

3.2 

51 

17.7 

29 

3.6 

GRADE LEVEL 

5th Grade 

70 

22.9 

20 

1.9 

0 

0 

3 

0.4 

6th Grade 

105 

34.3 

248 

23.1 

97 

33.6 

199 

24.6 

7th Grade 

70 

22.9 

397 

37.0 

74 

25.6 

307 

38.0 

8th Grade 

48 

15.7 

382 

35.6 

63 

21.8 

282 

34.9 

Other 

13 

4.3 

26 

2.4 

55 

19.0 

17 

2.1 


What are Students’ Patterns of Participation in Elev8 OST? 


As can be seen in Figure ES-i, across all regions, about four 
in ten students in middle grades in Elev8 schools 
participated in Elev8 OST activities at least once during the 
2010-11 school year. 8 However, participation varied by 
region. This is due, in part, to differences in program focus. 


About four in ten middle-grades 
students in Elev 8 schools 
participated in Elev 8 OST activities 
at least once during the 2010-11 
school year. 



8 Grades targeted for Elev 8 are primarily the middle grades: 6 th , 7 th and 8 th . However, there is variation by school. In Baltimore, for instance, 
some 5 th graders are included in Elev 8 because the school configuration is K- 8 . In Chicago, there is a school that only serves 7 th and 8 * graders. 
In Oakland, some campuses include 9 th graders and therefore some 9 th graders receive programming. Data are from 2011 Elev 8 participation 
summary. 
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• Participation in days: Across regions, students participating in Elev8 OST attended the 
program for an average of 43 days, ranging from 1 to 143 days in 2010-2011. This compares 
favorably to the average number of days students in Providence’s After Zone participated in OST 
programming. 9 9 10 

• Participation in months: Students participating in Elev8 OST did so for 5.3 months of the 
school year on average. Across regions, participation averages ranged from 3.8 to 6.3 months. 

• Breadth of participation in activities and services: Only one region had data on the types 
of Elev8 activities students engaged in; on average, students in this region participated in two to 
three different types of Elev8 OST activities. Across all regions, just under a quarter of students 
in Elev8 schools both participated in Elev8 OST and took advantage of the school-based health 
center, another key Elev8 service. 

How Do the Demographic Characteristics of Participants and Non-Participants 
Compare? 

Across regions there are differences between students who 
opted to participate in Elev8 OST activities for at least one 
day and those who did not. Students who participated in 
Elev8 OST are more likely to be African American and less 
likely to be Hispanic than non-participants. They are also 
more likely to be in the younger grades than their non-OST 
participating peers. There are no statistically significant 
differences in gender or income status between Elev8 OST 
participants and non-participants. 


Students who opted to participate in 
Elev8 OST were more likely to be 
African American and to be in 
younger middle school grades than 
those who opted not to participate. 


How Is Participation in Elev8 OST Related to Students’ Experiences of 
School? 

This evaluation examines the relationship between student participation in Elev8 OST programs and 
two school experience outcomes we had access to in our data: 8 th grade students’ high school transition 
planning and attitudes toward school. After taking into account pre-existing differences in student 
r ace/ ethnicity, gender, student socioeconomic status, and schools students attended, our analyses 
suggest the following: 

• Among those who participated in Elev8 OST, higher levels of participation are associated with 
more positive high school planning outcomes. More specifically: 

s Students who attended more days in Elev8 OST were more likely to participate in a wider 
range of high school planning activities. 

s Engaging in more high school planning activities is associated with a greater likelihood of 
having a plan for high school. 


9 In the 2008-09 school year, middle schoolers in the After Zone program participated for an average of 24 of the 96 days of OST programming 
available to them. Over 80 percent participated in no more than half of the available days (Kauh, 2011). 

10 See, for instance, Little, P.M.D & Harris, E. (2003). A Review of Out-of-School Time Program Quasi-Experimental and Experimental 
Evaluation Results. In Out of School Time Evaluation Snapshots. Number 1. Harvard Family Research Project. Kauh, T. (2011). AfterZone: 
Outcomes for Youth Participating in Providence’s Citywide After-School System. Philadelphia: Public/Private Ventures. 



S Students who attended more days of Elev8 OST are more likely to report planning to apply 
for a competitive college preparatory high school (defined as a selective enrollment— public 
or private— or charter high school). 11 

• The Elev8 program is a whole-school model and as such it might benefit all students who 
attended Elev8 schools, including OST participants and non-participants. According to our 
analysis: 

■/ Both Elev8 OST participants and non- 
participants reported high levels of efficacy 12 , 
liked schools they attended, and valued school 
highly. 

■S More than two thirds of students in Elev8 
schools (including OST participants and non- 
participants) reported that they feel school is 
relevant to their future. 

■S The majority of the 8th-grade students in 
Elev8 schools (including OST participants and 
non-participants) participated in high school 
planning activities and reported having clear 
high school plans. 

Our study was not designed to determine definitively if 
participation in Elev8 OST is the cause of these 
outcomes. Therefore, these findings do not definitely 
demonstrate that Elev8 OST participation is beneficial to 
students’ high school planning efforts beyond the benefit 
Elev8 may exert school-wide (irrespective of 8 th graders’ 
participation in Elev8 OST). Yet, they do suggest that 
increased participation levels in Elev8 OST is associated 
with better high school planning efforts and higher high school aspirations within the subgroup of 
Elev8 OST participants. These are relationships built into the Elev8 initiative to help students establish 
a model pathway to postsecondary success (see Figure ES-2). 


Eighth grade Elev8 participants who 
attended more frequently were 
significantly more likely to 
participate in a wide range of high 
school planning activities (such as 
spending the day shadowing at a 
high school of interest, attending a 
test preparation class, attending a 
mock high school interview, etc.) 
than those with lower OST 
participation. Also, Elev8 OST 
participants who participated more 
frequently were more likely to report 
that they planned to attend a 
college-preparatory (selective 
enrollment— public or private— or 
charter) high school than those with 
less frequent participation. Elev8 
OST participation was not related to 
school attitudes, possibly because 
students had uniformly positive 
attitudes towards school. 


11 Regions varied in terms of the availability of selective enrollment and charter high schools. Some regions have robust options, while others 
have fewer choices. One school had only one feeder high school. Finally, we do not have data to confirm that these schools actually have higher 
rates of students attending or prepared for college than traditional neighborhood schools. 

12 Their perception that they can achieve in school. 
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Figure ES-2: Theory of action between participation in Elev8 OST and participants’ high school plans 



Final Thoughts 

Despite limitations, these findings suggest that: l) increasing participation levels in OST programs in 
community school settings may promote OST participants’ success in high school; and 2 ) Elev8 
students in our sample have positive attitudes toward school. Elev8 aims to guide students through the 
steps necessary to prepare, plan for, and attend high-quality high schools through various activities 
taking place in the school day and by bolstering these school-wide activities during OST programming. 

As Elev8 and other community schools strive to meet their central goal of preparing students for high 
school, OST activities may be a promising context to help inform students about their high school 
options and planning activities. More research is needed to understand the relationship between Elev8 
OST and positive student outcomes. 
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Chapter I: Introduction 

Far too many children across America face substantial barriers to educational success. Poverty and lack 
of access to high-quality learning opportunities and healthcare mean young people do not receive the 
support they need to thrive. A solid, high-quality education is a key to successful adulthood; however, 
young people from low-income families and communities often face a myriad of educational and 
academic challenges that put them at risk for ongoing disadvantage. Community schools are an 
approach that has been adopted to respond to this educational crisis. Community schools are 
educational models that pool resources and services to meet students’, families’, and communities’ 
needs in order to alleviate systemic problems associated with poor performance in school. By 
partnering with local agencies, community schools provide students and families with access to 
supports to increase student outcomes and overall quality of life, such as healthcare services; child care; 
educational enhancement and recreational opportunities; family economic supports, such as workforce 
development and income tax assistance; and other programming, like educational advocacy activities, 
leadership opportunities, child care, and others. 13 Proponents of the model posit that community 
schools’ “integrated focus on academics, health and social services, youth and community development 
and community engagement leads to improved student learning, stronger families and healthier 
communities.” 14 

In 2008, The Atlantic Philanthropies (Atlantic) made an investment in its first five community schools 
in the State of New Mexico. Over the years that followed, Atlantic invested in developing community 
schools in three additional locations where there was also the need for support of low-performing 
middle schools and middle-school students— Chicago, Baltimore, and Oakland— totaling about 20 
schools across all four regions. A brief look at the communities where the investments were made shows 
some of the reasons why they were targeted: 

• In the City of Chicago, there is a 50% high school graduation rate, compared with 70% 
statewide. 15 

• In the State of New Mexico, over 25% of children live in poverty. 16 


'3 Bireda, S. (2009). A Look at Community Schools. Center for American Progress. Retrieved November 29, 2012 from 

http://www.americanprogress.org/wp-content/uploads/issues/2009/10/pdf/community_schools.pdf 

*4 What is a Community School? Coalition for Community Schools. Retrieved November 29, 2012, from 

http://www.communityschools.org/ahoutschools/what_is_a_community_school.aspx 

'» Source: Illinois Department of Education, 2010 

16 Source: American Community Survey (2009) 


• In the City of Baltimore, 50% of public school 8 th graders scored in the “basic” category on the 
Maryland School Assessments in reading and math. 

• In the City of Oakland, 21% of children live below the poverty line. 18 

For this initiative, now known as Elev8, 

Atlantic selected agencies with deep local roots 
to serve as regional leads. Each of these 
grantees has developed and implemented 
flexible, full-service community school models 
in up to five schools, focusing on middle-grade 
students in low-performing schools in their 
region. Their efforts build on core elements 
that research has linked with student 
achievement and success. ltJ These include 
extended-day learning opportunities, school- 
based healthcare, and support for families, 
which are all integrated into the school to make 
good use of resources and effectively support 
students. 

About this Study 

Assessing the impact of efforts like Elev8 
requires time and patience— measurable 
results take time to achieve and are difficult to 
prove definitively. In recognition of this, since 
2008, Atlantic’s evaluation effort has focused 
on generating information to help create and 
sustain the strongest initiative possible. As a result, the first five years of the evaluation were designed 
to ensure that the model was robustly implemented, and to assist the local sites in utilizing evaluation 
information to strengthen their efforts. 

Atlantic and their grantees are now seeking to document how students fare in the program, and to 
contribute to the literature on community schools. With this in mind, the evaluation team set forth to 
explore what questions we could answer with the data we had in hand. 

Our review of the literature suggests that, despite the fact that Out-of-School Time (OST) activities are a 
fundamental aspect of the community school model, little is known about the impact of OST activities 
on students in the context of community schools. Research has shown that participation in high-quality 
OST programs can provide important opportunities for disadvantaged youth and can result in 
significant social and educational gains, including: improved school attendance; stronger academic 
performance; better attitudes towards and behavior in school, improved relationships; and, decreased 


Atlantic and their grantees are now seeking to 
document how students fare in the program, 
and to contribute to the literature on 
community schools. With this in mind, the 
evaluation team has used data originally 
intended as part of the program improvement 
effort to achieve these goals. The result of that 
effort is presented herein. 


This study answers the following questions: 

• Who participates in OST in Elev8 schools? 

• What are their patterns of participation? 

• How do Elev8 OST participants compare 
to students in Elev8 schools who do not 
participate in Elev8 OST (non-OST or non- 
participants in this report)? 

• How is participation in Elev8 OST related 
to students’ experiences of school? 


17 Source: Kids Count (2012) 

18 Source: Oakland Unified School District, CA: DP3 Economic Characteristics. ProximityOne. Retrieved from 
http://pr0ximity0ne.c0m/acs/dpca/dp3_0628050.htm 

Blank, M.A., Melaville, A., & Shah, B.P. (no date). Making the difference: Research and practice in community schools. Coalition for 
Community Schools. Retrieved January 7, 2013 from http://www.communityschools.0rg/assets/1/Page/CCSFullReport.pdf. 
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delinquency. 20 Research also shows that participation in OST in a school-based setting can increase 
parental involvement in school. 21 By design, community schools should be able to enhance these 
outcomes as they are likely to bolster youth’s participation in OST activities by reducing barriers to 
participation such as: transportation; peer involvement; developing programming that is responsive to 
the school’s and students’ needs; and, reducing participation reluctance or hesitancy (as teachers and 
school staff can collaboratively encourage youth to participate 22 ). 

As a significant investment is made in OST activities in Elev8 and other community school models, this 
report takes a preliminary look inside the “black box” of community schools, thereby providing 
important information to the community schools field about the relative value of OST within a 
community school model. This study examines the relationship between student participation in 
community schools’ OST activities and students’ experiences in schools that may be related to more 
positive academic outcomes. This study answers the following questions: 

• Who participates in OST in Elev8 schools? 

• What are their patterns of participation? 

• How do Elev8 OST participants compare to students in Elev8 schools who do not participate in 
Elev8 OST (called non-OST or non-participants in this report)? 

• How is participation in Elev8 OST related to students’ experiences of school? 

To answer these questions, we used data from three main sources: 

• Administrative records containing student demographic data; 

• OST participation data (the “participation data”); and, 

• Self-report data from our annual survey of students in Elev8 schools. 

More details on the data sources and survey administration can be found in the appendices. 

It is critical to keep in mind that, while OST activities are a central component of Elev8, they are only 
one component of the initiative. Elev8 is a whole-school model, and all students in Elev8 schools have 
the opportunity to participate in school-wide Elev8 activities, such as health fairs and opportunities for 
increased parent and community involvement in schools, and will be exposed to changes to school 
climate and school policy brought about by the community school effort. Our analysis suggests that, on 
average, both Elev8 participants and non-participants reported: high levels of efficacy (their perception 
that they can achieve in school); liked schools they attended; and, valued school highly. Further, a 
majority of the 8 th grade students in Elev8 schools, including OST participants and non-participants, 
participated in high school planning activities and reported having clear high school plans. Notably, our 
analysis suggests that participation level in Elev8 OST activities is associated with planning and 
preparing for high school among 8 th grade Elev8 participants. 

In the sections that follow, we provide background information on the community schools model, its 
origin, and a brief review of the literature on community schools initiatives. We then provide a 
description of the Elev8 initiative, followed by information about the data sources used in our analyses. 


20 See, for instance. Little, P.M.D & Harris, E. (2003). A Review of Out-of-School Time Program Quasi-Experimental and Experimental 
Evaluation Results. In Out of School Time Evaluation Snapshots. Number 1. Harvard Family Research Project. Kauh, T. (2011). AfterZone: 
Outcomes for Youth Participating in Providence’s Citywide After-School System. Philadelphia: Public/Private Ventures. 

21 Grossman, J.B. & Vang, Z. (2009). The Case for School-Based Integration of Services: Changing the Ways Students, Families and 
Communities Engage With Their Schools. Philadelphia: Public/Private Ventures. 

22 Ibid. 
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The final sections of the report describe OST participation in Elev8 and present early findings on the 
relationship between Elev8 OST participation and student outcomes. 


Chapter 2: Community Schools and OST — What We Know 

History of Community Schools 

The community schools model began in the early 20 th 
century with John Dewey and Jane Addams in 
response to the need to provide immigrant children 
with important medical services, such as immunization 
and dentistry. * 2 s Today, in response to the 21 st century 
educational issues facing our nation’s most vulnerable 
youth, community schools exist in forty-nine states and 
the District of Columbia. 2 4 While community schools 
vary in terms of exact configuration of services and 
partners, according to the Children’s Aid Society, they 
share the following framework, known as the 
developmental triangle. 2 s Within this framework, 
common services include OST (including summer) 
programming, family support services, and services to 
minimize student barriers to success, such as school- 
based health, behavioral health, and dental services. The 
framework recognizes that the engagement of students, 
their families, and community members in the school and 
the community is central to student success and the strength of the educational system as a whole. 
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Evidence of Community Schools’ Success 

While community schools have existed for almost a century, research on the impact of this educational 
model is limited, but growing. Nonetheless, some recent research studies suggest that community 
schools do have a positive impact on academic performance, behavior, and parental involvement. The 
results on each of these indicators are summarized below. 

Improved Academic Performance 

Educational success is a central goal of the community schools movement, and research on community 
schools suggests that community school students experience gains in school. Research on the academic 
impacts of community schools tends to focus on test scores, attendance, and drop-out rates. An 
evaluation of the Communities in Schools (CIS) model found that case managers played an important 
role in bolstering academic achievement for 6 th graders, who achieved significant increases in their 


2 3 Dryfoos, J.G. (January, 2002). Full-service community schools: Creating new institutions. Phi Delta Kappan. 

2 4 Coalition for Community Schools (2009). Community schools: Research brief. Retrieved January 8, 2013 from 
http://www.communityschools.0rg/assets/1/AssetManager/CCS%20Research%20Report2009.pdf 

2 s National Center for Community Schools (no date). FAQs on Community Schools. Retrieved from 
http://nationalcenterforcommunityschools.childrensaidsociety.org/faqs/on-community-schools 


reading test scores; and 9 th graders, who achieved significant increases in grade point average. 26 An 
evaluation of the Children’s Aid Society (CAS) also demonstrated that students who participated in their 
after-school programs earned significantly higher test scores than students in other schools. 2 ? In a 
review of 49 evaluations of community schools, Dryfoos found that 36 evaluations reported academic 
gains: most of the evaluations that reported gains were in the elementary school grades, and 
improvements were limited to students who needed “specialized services, such as case management, 
intensive mental health services, or extended day sessions.” 28 The extended day finding is especially 
noteworthy, as many community school models include an extended day component, while fewer 
include case management. Research also suggests that students in community schools had increased 
attendance rates and decreased drop-out rates. 215 

Improved Behavior 

Community schools also strive to create a learning environment that fosters improvements in student 
behavior. Several evaluations have suggested that suspension rates and disciplinary incidents are lower 
in community schools and among students attending community schools, though the evidence 
supporting these outcomes is somewhat mixed. In Dryfoos’ examination, 11 programs reported 
decreases in suspension rates. However, she posits that this decrease “may reflect changes in 
suspension policies, rather than changes in behaviors leading to suspensions. ”3° Chicago Public Schools 
manages the Chicago Community Schools Initiative (CSI), which is the largest community schools 
system in the country. 3 1 According to the Community Schools Research Brief, this system of schools 
consistently reported lower levels of disciplinary incidents compared to other schools in the district 
with similar student demographics. 32 However, while the five-year evaluation of CSI reported that 
school disciplinary incidents and suspensions decreased for students by the end of 6 th grade, high 
school students’ disciplinary incidents and suspensions increased. 33 

Increased Parental Involvement 

Many community schools evaluations have found that parental involvement increases over time. 
Indicators of parental involvement include attendance at parent -teacher conferences and volunteering 
at school events. Home visits conducted by Family-School Liaisons and opportunities provided for 
parents to volunteer or take classes may be associated with parents’ increased involvement in 
community schools. 

The Role of OST 

OST activities are a central component of community schools. Often, community schools are open for 
extended hours, providing students with a safe place to spend after-school hours and offering 


26 IFC International (2010). Communities in schools National Evaluation: Five Year Study. Retrieved November 29, 2012 from 
http://www.communitiesinschools.org/about/publications/publication/five-year-national-evaluation-summary-report 
2 ? Summary of the Children’s Aid Society community schools: Results to date; Referenced from 21 st Century Community Learning Centers at 
six New York City middle schools: Year one findings, prepared by K. Krenichy, H. Clark, N. Schaefer-McDaniel, & L. Benitez of ActKnowledge, 
September 2004. See also Coalition for Community Schools’ Community Schools: Research Brief (2009). 

28 Dryfoos, J.G. (2000). Evaluation of Community Schools: Findings to Date. Coalition for Community Schools. 

2 9 Coalition for Community Schools’ Community Schools: Research Brief (2009); Dryfoos (2000). Evaluation of Community Schools: Findings 
to Date; Summary of the Children’s Aid Society community schools: Results to date; and K. Krenichy, H. Clark, N. Schaefer-McDaniel, & L. 
Benitez of ActKnowledge, September 2004. See also Coalition for Community Schools’ Community Schools: Research Brief (2009). 

3 ° Dryfoos, J.G. (2000). Evaluation of Community Schools: Findings to Date. Coalition for Community Schools. 

3 1 Chicago Public Schools website. Retrieved on November 29, 2012 from 
http://www.cps.edu/Programs/DistrictInitiatives/Pages/CommunitySchoolsInitiative.aspx 

32 Coalition for Community Schools’ Community Schools: Research Brief (2009) retrieved November 29, 2012 from 
http://www.c0mmunitysch00ls.0rg/assets/1/AssetManager/CCS%20Research%20Rep0rt2009.pdf 

33 IFC International (2010). Communities in Schools National Evaluation: Five Year Study. Retrieved November 29, 2012 from 
http://www.communitiesinschools.org/about/puhlications/puhlication/five-year-national-evaluation-summary-report 
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programming targeted at increasing academic skills and positive youth development. The literature on 
the impact of school- and/ or community-based OST programs is robust, but mixed. Many programs 
boast increased academic achievement, positive youth development, and behavioral change, such as 
reductions in drug use (or initiation of drug use). 34 However, other research reports mixed effects of 
OST programs. For example, a 2003 evaluation of the federal government’s 21 st Century Community 
Learning Centers Program found no improvements in students’ behavior, feelings of safety, or 
developmental outcomes, including students’ reports of their ability to plan, set goals, or work with a 
team. 33 Additionally, this evaluation found no differences in academic outcomes between participants 
and the comparison group of non-participants. On a more positive note, the 21 st Century evaluation did 
find that parental involvement, a key to academic performance, increased for middle school 
participants; parents of middle school participants were more likely to volunteer at school, attend open 
houses, and help their children with homework. 

Documenting and measuring the associations between youth outcomes and activity participation is 
challenging, which may contribute to the contradictory findings in the literature. There are a myriad of 
OST programs available to youth and they vary by scale, type, affiliation, focus, and funding. 36 OST 
programs’ goals and content vary, particularly around academics, enrichment, recreation, and social 
support. 37 Recreational OST programs are unlikely to achieve positive academic outcomes for 
participants. Second, there is a great deal of variance in the amount and quality of time that youth 
spend in OST programs. 38 In a study of these programs in education settings, “only 16 percent of 
children ages 6-9 and 10 percent of children ages 10-12 attended [school-based or community-based 
after-school] programs for five or more hours a week.” 3 ? Research suggests that participation rates can 
vary in response to student motivation; 40 the needs of the family, whether that includes quality 
childcare or academic support; and program availability. 41 Students whose participation is of short 
duration or low frequency are unlikely to benefit from OST programming in meaningful ways. 


Chapter 3: The Elev8 Community Schools Initiative 

Elev8 is a community schools model operating in low-income areas, with the goal of alleviating the 
systematic problems associated with poor performance in school. Building off of the recognized best 
practices of the community schools approach, Elev8’s “wrap-around” services include: 

• Extended learning opportunities outside the classroom and beyond the traditional school year; 

• Parental outreach and engagement; 

• Accessible health services to students and families; and, 

• Resources for household economic stability. 


34 Weiss, H.B. (2004). Understanding and Measuring Attendance in Out-of-School Time Programs. Issues and Opportunities in Out-of-School- 
Time Evaluation. Harvard Family Research Project. Number 7. 

35 Mathematica evaluation of 21 st century: Dynarski, M. et al. (2003). When Schools Stay Open Late. U.S. Department of Education. Retrieved 
from http://www.mathematica-mpr.com/pdfs/whenschools.pdf 

3 6 Little, P.M.D., Wimer, C., & Weiss, H.B. (2008). After School Programs in the 21 st Century: Their Potential and What it Takes to Achieve It. 
Issues and Opportunities in Out-of-School-Time Evaluation. Harvard Family Research Project. Number 10. 

37 Halpern, R. (2002). A different kind of child development institution: The history of after-school programs for low-income children. 
Teachers college Record, 104(2), 178-211. Hynes, K., Smith, E.P., & Perkins, D. (2009). Piloting a school-based intervention in after-school 
programs: A case study in science migration. Journal of Children’s Services, 4, 4-20. 

3 8 Little, P.M.D., Wimer, C., & Weiss, H.B. (2008). After School Programs in the 21 st Century: Their Potential and What it Takes to Achieve it. 
Issues and Opportunities in Out-of-School-Time Evaluation. Harvard Family Research Project. Number 10. 

39 Hynes, K. & Sanders, F. (2010). The changing landscape of afterschool programs. Afterschool Matters (June), p.25 

4 ° Bodilly, S.J. & Beckett, M.K. (2005). Making Out-of-School-Time Matter. RAND Research Brief Education. Retrieved from 

www.rand.org/publications/MG/MG242 

4 1 Hynes, K. & Sanders, F. (2010). The changing landscape of afterschool programs. Afterschool Matters (June). 


6 


In addition to teachers and administrators, Elev8 schools employ a team of out-of-school time staff, 
family advocates, medical professionals, a site director, and others as dictated by their model. 

A Brief History of Elev8 

Elev8, initially called Integrated Services in Schools, was launched in 2007 with funding from The 
Atlantic Philanthropies. Elev8 founders had a national selection process to ascertain which 
communities could most benefit from Elev8’s integrated approach. Elev8 concentrates its efforts on the 
middle school grades, since research shows student engagement in middle school proves an accurate 
indicator of high school and post-graduate success. 42 The transition from 8 th to 9 th grade is particularly 
perilous, and a significant decline in academic performance upon reaching high school is associated 
with a greater risk of not completing college. 43 

Atlantic’s first investment in Elev8 was in three locations in New Mexico: Albuquerque, Laguna Pueblo, 
and Anthony, a rural border community in southern New Mexico. Three additional regions have 
received investments from Atlantic: Chicago, IL, Oakland, CA, and Baltimore, MD. As of this writing, 19 
19 Elev8 schools serve over 5,000 students in the middle grades. Approximately 2,500 students are 
served in Elev8 OST activities each year. During the 2010-11 school year, over 700 adult family 
members received Elev8 services. 

Elev8 Sites: A National Model with Local Customization 

While Elev8 is operating nationally in diverse parts of the 
country, Elev8 community schools function on a decidedly 
local level. Elev8 endeavors to address the specific needs of 
each community by working together with existing social 
infrastructure and institutions. Across Elev8’s four local 
initiatives, each has its own distinct format and unique path 
toward creating solutions. More detail, along with 
individual school profiles, is provided at 
www.Elev8kids.org. 

The Elev8 initiative assumes that the best solutions are often locally driven. For this reason, Elev8 looks 
different in each of the participating schools, 44 with community members identifying local needs and 
opportunities. Elev8 engages students and families so that, over the long term, they can be successful 
advocates for themselves and their communities. Elev8 is a multi-agency, multi-site project, with over 
75 contributing partners nationally. Table 1 provides a brief description of each region. 


A note about nomenclature 

In order to preserve the anonymity 
oftheElev8 regions, schools, and 
study participants, we use an 
identifier (Region A, B, C, D)for 
each site in place of its name. 


■* 2 Wimberly, G.L. & Noeth, R.L. (2005). College readiness begins in middle school. ACT: Iowa City, LA. 

43 Smith, J.S. (2006). Examining the long term impact of achievement loss during the transition to high school. The Journal of Secondary 
Gifted Education, 17, 211-221. 

44 There were 20 schools in the 2010-11 school year, the year in which most of the data for this study was collected. One of those 20 sites is only 
included in the counts of participants and not included in other analysis, because we were unable to collect identified individual level-data 
from students in that school. Currently, there are 19 schools. 
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Table I. Description of Elev8 Regions 


Elev8 Chicago 


Elev8 Chicago seeks to transform the educational achievement and life outcomes of middle-grade 
students in five public schools, all in underserved areas of the city. Students at the Elev8 schools, 
generally from low- income Latino or African-American families, are burdened by an inadequate 
healthcare system, challenging social environments, and limited economic opportunities. The program is 
led by the Local Initiatives Support Corporation's Chicago office (LISC/Chicago), which organizes capital 
and other resources to support initiatives that stimulate the comprehensive development of healthy, 
stable neighborhoods. Elev8 Chicago is built on the platform of LISC/Chicago's New Communities 
Program. Each of the five participating Chicago public schools is partnered with a lead agency that works 
with scores of local organizations on quality-of-life issues including education, family supports, public 
safety, healthcare, and opportunities for youth. 


Elev8 Oakland 


Elev8 Oakland launched in September 2008 on five school campuses in the city’s flatlands— a series of 
densely populated neighborhoods characterized by extensive racial, ethnic and cultural diversity. Fifty 
percent of local residents speak a language other than English at home. Elev8 Oakland is a program 
of Safe Passages, founded in 1998 as one of five sites across the countiy selected to participate in the 
national Urban Health Initiative funded by the Robert Wood Johnson Foundation. Today, Safe Passages 
serves as an intergovernmental partnership including the City of Oakland, the County of Alameda, and 
the Oakland Unified School District, as well as philanthropic and community-based partners who are 
committed to advocating for children, youth, and families, with a special emphasis on vulnerable 
populations within the County of Alameda. 


Elev8 New Mexico 


Elev8 New Mexico is being implemented in five diverse middle schools statewide— in urban, rural, and 
tribal communities. Elev8 New Mexico is an initiative of Youth Development, Inc. (YDI), a statewide 
agency that has been serving the comprehensive needs of youth and families in New Mexico for over 40 
years. With the participation of a broad array of organizations, elected officials, students, families, and 
schools, Elev8 New Mexico is integrating extended-day learning programs, healthcare, family supports, 
and community engagement so students succeed in school and in life. 


Elev8 Baltimore 


Fall 2009 marked the launch of Elev8 Baltimore, which partners with four prekindergarten-8 th grade 
neighborhood schools in East Baltimore, a community comprised of several neighborhoods located near 
the Johns Hopkins University Medical Campus. Elev8 Baltimore is an initiative of the East Baltimore 
Development Inc. (EBDI), an innovative public-private partnership established in 2002 to facilitate one 
of the most comprehensive, groundbreaking community redevelopment efforts in America. Elev8 
Baltimore and its partners— including EBDI, the Annie E. Casey Foundation, the Baltimore Community 
Foundation, Johns Hopkins Medical Institutions— are committed to transforming school communities by 
expanding learning, health, and family economic supports to children, youth, and families. 
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Chapter 4: Participation in Elev8 OST 

Elev8 OST is an essential component of the Elev8 initiative. However, the exact nature of the 
programming varies by region and school. Some OST programs have an academic focus, concentrating 
on tutoring or homework help, while others center on recreational activities like cooking or dance 
classes. Elev8’s middle school-aged participants tend to “vote with their feet,” thereby establishing a 
critical link between program satisfaction and attendance. Most Elev8 initiatives strive to implement 
recreational activities with an academic focus in order to both engage and retain young people, while 
providing students with the educational supports they need. At the program level, all Elev8 schools 
provide both recreational and academically focused OST programming. 45 

Students come into the OST program in a variety of ways. In most cases, participation in Elev8 OST 
activities is open to all students who are interested. In a handful of schools, however, students 
struggling academically are exclusively targeted for Elev8 OST programming. Additionally, some Elev8 
schools offer non-Elev8 OST activities to students, so a student not enrolled in Elev8 OST may still have 
the opportunity to participate in other OST activities. 

In the section that follows, we describe Elev8 OST 
participants and their attendance patterns during the 2010- 
11 school year. The analyses (with the exception of the 
participation rate— Figure 1 below) are based upon Elev8’s 
programmatic database, which contains administrative, 
demographic and participation information on Elev8 OST 
participants. Later in this report, we use self-reported 
demographic data from school-wide student surveys. Survey 
data include information from all students, rather than only 
the subset of students who attended OST activities. 

While the two data sources mostly provide parallel 
information, there are a few discrepancies, which we note 
below. 

How Many Students Participate in Elev8 OST? 

Overall, about 40 percent of students in Elev8 schools take advantage of Elev8 OST activities. However, 
as shown in Figure 1, 46 the number of students that participated in Elev8 OST activities in the 2010-11 
school year varied considerably by site. This is in part due to differences in program focus. For instance, 
Region C targets students who are struggling academically. This region uses an Early Warning Indicator 
(EWI) framework to identify students who experience challenges in attendance, behavior, and 
academics (math or language arts), and enrolls them in Elev8 OST programming. Other regions strive 
to serve all interested students: OST participation in Elev8 schools in two regions is roughly 60%, while 
it is slightly more than one-third in the last region. These figures are generally comparable (one slightly 
lower and the other two slightly higher) to favorably to the percent of students who opted to participate 
in Providence’s After Zone (which also targets middle school students and is not a targeted program) 


A note about data 

The data in this section are from our 
attendance database (except for 
figure 1, which is from an internal 
report developed by the Elev8 
evaluation team). Later in the 
report, where we compare OST 
participants to non-participants, we 
use a more limited sample 
comprised of students who 
completed a student survey. As such, 
there are some discrepancies in 
numbers. 


-ts Only one region provided data that document the actual types of OST programs that students participate in. Those data are provided later in 
this report. 

4 6 The data in figure l are extracted from the 2011 Elev8 participation summary report— an internal formal report the evaluation team prepares 
for The Atlantic Philanthropies and the regions once a year. 
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OST programming in the 2008-09 school year (just under half of students). 47 Different levels of 
resources and other factors, such as grades served and activity quality, also likely influenced student 
participation in OST activities across regions and schools. 


Figure I. Elev8 OST participation rate by region and overall during the 2010-1 I school year 



Region A Region B Region C Region D All regions 

Percent of Students in Elev8 Schools Who Did Not Participate in Elev8 OST 
Percent of Students in Elev8 Schools Participating in Elev8 OST 


Racial and Ethnic Characteristics of Elev8 OST Participants 

For the most part, Elev8 OST participants are students of 
color. As displayed in Figure 2, however, the exact racial 
and ethnic compositions of participants vary by site. 
Specifically, in Region A the vast majority of OST 
participants are African American. Region B serves an 
almost equal number of African-American and Hispanic 
students. In Region C, the majority of youth who 
participate in Elev8 OST are Hispanic students and more 
than one-third are African American. Region C also has the 
largest percentage of students of Asian descent in their 
Elev8 OST programs. Though Region D serves the largest proportion of Native American students in its 
OST programs, Hispanic youth are the largest portion of their OST participants. 


Representative of the regions in 
which Elev8 is located, students who 
participate in Elev8 OST are 
primarily low income students of 
color. Female students and middle 
schoolers in the younger grades are 
most likely to participate. 


47 Kauh, T. (2011). AfterZone: Outcomes for Youth Participating in Providence’s Citywide After-School System. Philadelphia: Public/Private 
Ventures. 
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Figure 2: Race and ethnicity of Elev8 OST participants 


Region A Region B 





African American 
Native American 


Asian 

Other 


Hispanic/Latino 

White 




Gender 


Overall, Elev8 OST participants are more likely to be female than male. However, the gender 
breakdown varies by region, as shown in Figure 3. In Region C, the percentages of boys and girls who 
participate in Elev8 OST are about equal, but in Regions A, B, and D, more girls participate in Elev8 
OST than boys. 

Figure 3. Gender of Elev8 OST participants 


Region A Region B 




Female 



Male 
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Income Status 


The bulk of students who participate in Elev8 OST, as well as those who attend Elev8 schools, are from 
low-income homes. 48 In Region A, the data for Elev8 OST participants was unavailable from program 
administrative records; however, as we will see later in this report, almost all students from Elev8 in 
Region A self-report as receiving free or reduced-price lunch (FRL) at school. Region D serves the 
largest proportion of students who are not eligible for free or reduced-price lunch (non-FRL) (see 
Figure 4). 


Figure 4: Income status of Elev8 OST participants 


Region A 


Region B 




NON-FRL 



Not Sure or Missing 


48 we gauged income status by using free or reduced-price lunch eligibility as a proxy for income. 
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Grade Level 


Elev8 targets middle-grade students. Overall, the data show that younger students are more likely to 
participate than older students (see Figure 5). This condition holds true in many OST programs 
nationwide. However, notably, in two of the four sites at least one quarter of Elev8 OST participants 
were in 8 th grade or older. Region A had the lowest percentage of 8 th graders served, a situation they 
attribute to the low numbers of 8 th grade students enrolled at each school. Nonetheless, approximately 
16 percent of participants in OST in Region A were 8 th graders. 

Figure 5: Grade level of Elev8 OST participants 


Region A 


Region B 



What Are Students’ Patterns of Participation in Elev8 OST? 

Describing participation in OST activities is complex because there are a myriad of programs that fall 
under the category of OST. For instance, a school sport— a very intensive but short-term activity— can be 
considered an OST activity. School sports are most likely staffed by school employees, funded by the 
district, heavily regulated, and emphasizing physical activity. Homework help is also a common OST 
program offered to young people. Homework Help differs in that it may or may not be taught by school 
staff, it may run every day all school year long, it has limited regulations, it focuses on academics, and it 
may be funded by an outside agency or by the school itself. Meanwhile, a cooking class, most likely 
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staffed by a non-school staff person, could be of limited duration (occurring once or twice a week), not 
heavily regulated, focused on recreation, and funded by an outside agency. 


A student attending any of these programs is engaged in an OST program. Therefore, student 
experiences and levels of engagement are going to be very different across and within programs. For 
example, on a sports team, a few students may be selected as captains and take on leadership roles. 
Other students may be starters and play all the time. Finally, some students will play smaller roles on 
the team and will have more of an opportunity 
to learn through observation and practice. 

Because of this diversity of experiences, OST 
researchers have identified four main 
dimensions of OST participation: ever 
participated, intensity, duration, and breadth . 49 
We describe how we have operationalized each 
term below. 

• Ever participated. This measure of 
participation is the most commonly 
used, and reflects whether a young 
person ever participated in a program 
during the timeframe of interest. It is the 
most basic measure of program 
participation. 

• Intensity. Intensity is the amount of 
time youth attend a program during a 
given period and can be measured in the 
following ways: hours per day, days per 
week, weeks per month, or months per 
year. 

• Duration. Duration is the length of time 
youth attend a program and captures information about how “new” a young person is to the 
program. Whereas intensity describes the “depth” of attendance, duration addresses the history 
of attendance in months or years since the young person started in the program. 

• Breadth. Breadth of attendance refers to the variety of activities that youth attend within and 
across programs. For instance, while one young person may participate only in a homework help 
program, another may participate both in organized homework help and in a sports program. 
Breadth can also be measured within a single program; for instance, some OST programs aim to 
provide both academic support and recreation activities. 


Key Findings: Participation Patterns in 
Elev8 OST 

• Forty percent of students in middle grades 
in Elev8 schools participated in Elev8 OST 
in the 2010/2010 school year. 

• Across regions, students participating in 
Elev8 OST attended the program for an 
average of 43 days. 

• Many students participated for just a few 
days; however, a meaningful number 
attended with high frequency. 

• Students participating in Elev8 OST did so 
for 5.3 months of the school year on 
average. 

• On average, students participated in 2.3 
different types ofElev8 OST activities. 
Across all regions, just under a quarter of 
students in Elev8 schools both participated 
in Elev8 OST and took advantage of the 
school based health center. 


49 Weiss, H.B. (2004). Understanding and Measuring Attendance in Out-of-School Time Programs. Issues and Opportunities in Out-of-School- 
Time Evaluation. Harvard Family Research Project. Number 7. 
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Our ability to understand each of these facets of participation is limited in the current study because of 
the data available to us. However, we do have data that describe the following for each student in our 
sample: 

• Participation in Services. Whether s/he participated in Elev8 OST only, Elev8 school-based 
health services only, no Elev8 services, or in both of these Elev8 services. 

• Length of Participation (in Days): Whether s/he ever participated in Elev8 OST and how 
many days during the school year s/he participated in Elev8 OST. 

• Length of Participation (in Months). How many months of the school year s/he 
participated in Elev8 OST. 

• Participation in Activities. How many different types of Elev8 OST activities s/he 
participated in during the school year (Region B only). 

Below, we present how Elev8 OST participants engaged in the program on each of these dimensions 
during the 2010-11 school year. 

Length of Participation (in Days) 

As seen in Figure 1, about forty percent of students in Elev8 
schools attended at least one day of Elev8 OST during the 
2010-11 school year. 

Students across regions who participated at least once had 
an average attendance of 43 days over the school year, 
which compares favorably to the average number of days 
students in Providence’s After Zone participated in OST 
programming. 5° However, this statistic masks broad 
diversity— both across and between regions. 

Although attendance rates averaged more than 40 days for 
Elev8 OST participants, Figure 6 shows that more than half 
of the participants attended Elev8 OST programs for less 
than 50 days and less than a quarter participated in the OST 
program very intensively over the school year (greater than 
75 days). 


A note about how to read 
Figures 6 and 7 

The figures that follow show the 
percentages ofElev8 participants 
for the number of days they 
participated in Eleu8 OST. As Figure 
6 shows, 100 percent ofElev8 OST 
participants participated for one 
day. At about 30 days only half of 
participants were still active in 
Elev8 OST, and at about 145 days, 
no Elev8 OST participants were still 
active in the program. 


s° In the 2008-09 school year, middle schoolers in the After Zone program participated for an average of 24 of the 96 days of OST 
programming available to them. Over 80 percent participated in no more than half of the available days (Kauh, 2011). 
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Figure 6. Percentage of OST participants who ever attended Elev8 OST by the number of days they attended, across 
regions 



Figure 7 illustrates the differential Elev8 participation levels between regions. Students in Region A 
participated in Elev8 for the highest number of days across the school year, averaging 53 days a year. 
Students in Regions B and D had lower levels of participation, with an average of 36 days and 38 days, 
respectively. Finally, students in Region C had the lowest participation levels in Elev8 OST, averaging 
just 24 days a year. The graphs in Figure 7 below tell a story similar to the aggregate data presented in 
Figure 6 above. Each region has a large percentage of participants that attended less than 50 days in 
Elev8 OST programs and a smaller percentage of students with high levels of participation. 
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Figure 7: Percentage of OST participants who ever attended Elev8 OST by the number of days they attended by region 


Region A Region B 



Minimum Number of Days Attended 


Length of Participation (in Months) 

We measured monthly participation in Elev8 OST to estimate the amount of time that participants 
chose to engage in Elev8 OST activities over the school year (see Figure 8). Additionally, we calculated 
the average number of months participants remained active in Elev8 OST. On average, students who 
participated in Elev8 OST across all sites participated for 5.3 months of the school year. However, there 
was variation by region. In Region A, students attended Elev8 OST for the greatest number of months, 
averaging 6.3 months per year, including just over one third who participated for the entire school year. 
Region C’s Elev8 OST participants attended for the shortest amount of time, averaging 3.8 months, 
including almost one third who participated for only one month. 
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Figure 8. Percentage of students participating in Elev8 OST activities by number of months (school year 2010-1 I) 



Participation in Activities 

Some, but not all, regions record the types of activities in which OST participants engage. Across all 
regions, there were almost 150 different Elev8 OST activities recorded in the 2010-11 school year. We 
grouped these activities into the following activity types in order to explore the breadth of student 
participation in Elev8 OST: general afterschool, sports and fitness, music and dance, development and 
learning, leadership activities, hobbies, computers, arts and crafts, and other. Regions varied in their 
activity records. Two regions did not record activity type and a third region only offered one Elev8- 
funded OST activity at four of its five schools, though other non-Elev8 activities were available during 
OST hours. In the remaining region, Region B, students participated in an average of 2.3 activity types 
during the school year. In this region, about one third participated in only one activity, a quarter 
participated in two activities, 15 percent participated in three activities, 13 percent participated in four 
activities, and the remainder participated in more than four activities. 

Participation in Services 

Using the sample we created by matching students in Elev8 schools who completed a survey to the 
participation data, we also measured the extent to which students in Elev8 schools participated in more 
than one Elev8 service (see Table 2). Our data only allows us to measure participation in Elev8 OST and 
participation in the school-based health center, and we were not able to assess the extent to which 
particular students’ families accessed services through the family support component of the program. 
Nonetheless, we found that about a quarter of students in Elev8 schools did not participate in either the 
school -based health center or Elev8 OST; about one half participated in only one of the two Elev8 
services, either Elev8 OST or the school-based health center; and, the last quarter of students utilized 
both services. 
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Table 2. Distribution of student participation Elev8 services: school-based health center and OST 


Number of Services 

0 

1 

(either OST or SBHC) 

2 

(both OST and SBHC) 

% of Sample 

25.83% 

50.93% 

23.24% 


Chapter 5: How is Participation in Elev8 OST Related to Students’ 
Experiences of School? 


To see if participation in Elev8 OST is related to students’ experiences in school, this evaluation 
compared students in Elev8 schools who participated in Elev8 OST with those who did not participate; 
it also compared those who participated with different levels of frequency. While we controlled for some 
observed factors, because participation in OST in most schools was determined by student choice or 
through targeting rather than random assignment, we cannot rule out the possibility that participants 
vary from non-participants (or between high- and low-frequency participators) in important ways. For 
instance, students who chose to participate in Elev8 OST could be more motivated to pursue 
educational enrichment activities than those students who did not participate. And students who 
participate more frequently may be more motivated than those who participate less often. Alternatively, 
those students who chose to participate (or participated more) might have done so because they needed 
additional academic support whereas those who did not participate (or participated less) might not 
have needed such support. 

For these reasons, we cannot determine with certainty that the differences in the groups’ school 
experiences are due to Elev8 OST participation. There are other limitations to our study that may result 
in an underestimation of the relationship between Elev8 OST participation and school experiences. 

• Our analysis below is based on a data set we created by matching students in Elev8 schools who 
completed a survey to students in the OST participation database. The matching protocol 
utilized students’ names, schools, and regions, though students who use or go by more than one 
name (such as a nickname, or students who have hyphenated last names) may not have been 
matched properly, s 1 

• Students identified as having not participated in Elev8 OST may have participated in other OST 
activities that are not reported in our data.s 2 

• Elev8 OST is only one component of Elev8, and whole school benefits of Elev8 may wash out the 
relationship between participating in Elev8 OST and school experiences since we are comparing 
Elev8 OST participants with students in the same school (who have also been “exposed” to 
Elev8) who did not participate in Elev8 OST. 

• Other school factors, such as characteristics of teachers and principals, or other school activities, 
are deemed important for student academic performance and behavioral outcomes in the 
literature, but are not available for this study. 


s* Our protocol for matching utilized exact matching via Stata and then manual matching of imperfectly matching names by region and school. 
s 2 Other limitations include: analysis of high school plans and planning activities includes a smaller sample size of only 8 th grade students. 
Finally, school fixed effects were not uniformly controlled for because of the small sample size. 
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In the section that follows, we investigate how those who did participate in Elev8 OST compare 
demographically to those students who did not participate in Elev8 OST activities during the 2010-11 
school year. Then we examine how participation in Elev8 OST is associated with students’ experiences 
in school and 8 th graders’ high school transition. We conclude with a summary of our findings. While 
the data we use for these analyses is limited to students who completed a survey (a smaller number of 
Elev8 OST participants than that reported above), the ratio of participants to non-participants is similar 
to the larger sample (see Appendix B for more detail). 

Comparing the Demographic Characteristics of Participants and Non-Participants 

Across all regions there are differences between students 
who participated in Elev8 OST activities for at least one day 
and those who did not. Elev8 OST participants are more 
likely to be African American and less likely to be Hispanic 
than non-participants. Elev8 OST participants are also more 
likely to be in the younger grades than their non-OST 
participating peers. There are no statistically significant 
differences in gender and income status between Elev8 OST participants and non-participants. 

Since the regions vary in terms of student demographics, their OST models, and their resources, we also 
looked within each region to see how non-participants compared with OST participants. (For more 
detailed information on how Elev8 OST participants and non-participants compare on demographic 
characteristics, see Appendix D). While in some schools the characteristics of Elev8 OST participants 
and non-participants were similar, in other schools they were very different. In Region B, Elev8 OST 
participants were much more likely to be Hispanic, and much less likely to be African American than 
non-OST students. In Region D, Elev8 OST participants were more often Native American than non- 
OST students. In Region A, Elev8 OST participants were slightly less likely to receive free or reduced- 
price lunch than those who chose not to participate in Elev8 OST. In Region C, Elev8 OST participants 
were more likely to be male than non-OST students, but in Region D Elev8 OST participants were more 
likely to be female. Finally, there were some differences in the proportion of students in grades 5, 6, 7, 
or 8 between Elev8 OST participants and non-OST students within the same regions. 

How Is Student Participation in Elev8 OST Activities Related to School Experience and 
8 th Graders’ High School Transition? 

In this report we strive to ascertain if students who participate in Elev8 OST have more positive school 
experiences than students who attend Elev8 schools but do not participate in Elev8 OST. To conduct 
this analysis we compare the two groups’ end-of-school-year reports for the 2010-11 school year. We 
also analyzed the data from Elev8 OST participants only to determine if more frequent participation in 
Elev8 OST is related to school experience. 


Students who participated in Elev8 
OST were more likely to be African 
American and to be in younger 
middle school grades than those who 
did not participate. 
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Overall, our analysis suggests that 8 th grade Elev8 OST participants who attended more frequently were 
significantly more likely to participate in a wide 
range of high school planning activities (such as 
spending the day shadowing at a high school of 
interest, attending a test preparation class, 
attending a mock high school interview) than 
participants with lower OST participation. Also, 

8 th grade OST participants who attended Elev8 
OST more frequently also were more likely to 
report that they planned to attend a college 
preparatory (selective enrollment— public or 
private— or charter) high school. Elev8 OST 
participation was not found to be related to 
school attitudes, but this may be due to students 
having uniformly positive attitudes towards 
school. 

Was Participation in Elev8 OST Related to 
Attitudes Toward School? 

Elev8 aims to improve students’ experiences in 
school and bolster school climate. We had access 
to data on the following components of a 
student’s attitude towards school: 

• Perception of their ability to achieve in school (school efficacy) 

• How much they like school (school liking) 

• How much they value school (school value) 

• Their perception of the connection between school and future (future connectedness) 

In order to assess if participation in Elev8 OST brings extra benefits to students above and beyond any 
school-wide benefits to both OST participants and non-participants, we first defined participation at the 
most basic level: participation in an Elev8 OST activity for at least one day over the school year. As 
shown in Figure 9, the differences in student attitudes towards school between Elev8 OST participants 
and non-participants are very small. Both Elev8 OST participants and non-participants reported high 
levels of efficacy and school value. On a scale from 1 (representing very low efficacy) to 4 (representing 
very high efficacy), student attitudes toward school in both groups averaged over 3. Similarly, both 
groups reported an average score of 3.3 (where 1 is very low and 4 is very high) on school value, 
indicating they valued school highly. Ratings of both school liking and future connectedness were also 
relatively high. On average students rated their school liking at a 2.8, while a majority of students 
reported that they feel school is relevant to their future. The aforementioned school liking and school 
value scores are comparable to scores reported in other studies of youth who chose to participate in 
OST activities. 53 


Key Findings: Elev8 OST participants 
and experiences in school 

Eighth grade Elev8 OST participants who 
attended more frequently were significantly 
more likely to participate in a wide range of 
high school planning activities (such as 
spending the day shadowing at a high school of 
interest , attending a test preparation class, 
attending a mock high school interview) than 
participants with lower OST participation. 

Also, 8 th grade Elev8 OST participants who 
attended Elev8 OST more frequently were more 
likely to report that they planned to attend a 
college preparatory (selective enrollment- 
public or private— or charter) high school. 

Elev8 OST participation was not found to be 
related to school attitudes, but this may be due 
to students having uniformly positive attitudes 
towards school. 


53 For instance, a 2008 study of 7 th and 8 th graders who attended Boys & Girls Clubs found that teens, on average, reported 2.76 on school 
liking and 3.39 on school value (school importance, revised). 
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Figure 9: Facets of attitude towards school by Elev8 OST participation status 


School Efficacy 


Future Connectedness 



Elev8 OST Non-OST 



Elev8 OST Non-OST 


Strongly Disagree bh Agree mmi Disagree Strongly Agree 


School Liking 
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School Value 


3.3 3.3 
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We conducted a set of more sophisticated analyses to rule out the effects of a range of pre-existing 
differences that might be relevant to student attitudes towards school: student demographics, such as 
student socioeconomic status, gender, race/ethnicity; grade level; years student stayed in a school; and 
unobserved differences in the schools that students attended. After taking into account these pre- 
existing differences in students and schools, we conclude that there are no significant differences in 
attitude towards school between Elev8 OST participants and non-participants. 

Following the work of other researchers, we also conducted a series of analyses investigating whether 
the participation levels in Elev8 OST are related to student attitudes toward schools. After taking into 
account pre-existing differences in student demographic characteristics and schools, our analyses do 
not provide any consistent evidence showing that participation levels in Elev8 OST are associated with 
student attitudes towards school. In other words, students who participate in Elev8 OST for more days 
of the school year are not more likely to report more positive attitudes towards school then those who 
participated less or not at all. However, it is important to note that the average of these scales may mask 
variation and therefore is not optimal for dosage analysis, which requires a higher level of precision. 

Was Participation in Elev8 OST Related to High School Planning? 

Elev8 aims to support middle grade students in order to ensure their successful transition to 9 th grade. 
We know that in the current labor market, college graduates earn significantly more than individuals 
without college degrees. As such, earning a college degree is a key pathway out of poverty. Therefore, 
high school completion is an important stepping stone and a meaningful credential. Unfortunately, 
despite recent progress, there remains a significant disparity in high school graduation rates among 
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young people from disadvantaged backgrounds. 54 Research has shown that the path to high school 
dropout begins before 9 th grade, and that the successful completion of 9 th grade is a critical step to high 
school graduation. 55 

To combat this trend, Elev8 supports students in their high school planning efforts, and encourages 
students to attend higher-performing high schools that will best support them in their secondary and 
postsecondary pursuits. While in many Elev8 communities neighborhood public schools struggle with 
violence, low attendance, high dropout rates, and minimal academic rigor, almost all of the districts in 
which Elev8 operates have some form of school choice, and students have the opportunity to attend 
higher-performing high schools if they take the necessary steps. Elev8 aims to guide students through 
the steps necessary to prepare, plan for, and attend high-quality high schools through various activities 
taking place in the school day, and by bolstering these school-wide activities during OST programming. 
For instance, in two regions Elev8 hosts school-wide high school fairs where students can learn about 
their high school choices, meet high school staff, and learn more about enrollment and entrance 
procedures. In one region, Elev8 OST staff is comprised of local college students who talk about their 
academic trajectories, their career plans, and their college experiences in their day-to-day interactions 
with Elev8 participants. These relationships are built into the Elev8 initiative to help students establish 
a model pathway to postsecondary success (see Figure 10 ). 

Figure 10: Theory of action between participation in Elev8 OST and participants’ high school plans 



To examine the relationship between 8 th graders’ participation in Elev8 OST and their high school 
transition, we provide some descriptive analyses showing how Elev8 OST participants are different 
from non-participants in their high school preparation efforts and their high school plans. This analysis 
is limited to 8 th graders, as high school planning questions were only asked of 8 th grade students. 

Elev8’s high school planning activities include high school visits, collecting information about specific 
high schools, test preparation classes, mock interviewing, meeting with students or teachers who attend 
or teach at a high school of interest, and more. 

Across all regions, students who participated in Elev8 OST reported engaging in an average of 2.0 types 
of planning activities, while students who did not participate in Elev8 OST reported engaging in an 


54 Balfanz, R. & Legters, N. (2004). Which High Schools Produce the Nation’s Dropouts? Where Are They Located? Who Attends Them? 

Report 70. Johns Hopkins University. Retrieved from: http://www.csos.jhu.edu/crespar/techReports/Report70.pdf 

55 Neild, R. & Balfanz R. (2006). Unfulfilled Promise. Project U-Turn, Philadelphia Youth Network; Philadelphia. 
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average of 1.7 planning activities. 56 Our descriptive analysis shows that students who participated in 
Elev8 OST engaged in a wider range of activities than those who did not participate. For example, as 
shown in Figure 11, lower percentages of Elev8 OST participants reported no high school planning 
activities than non-participants, and the percentages of Elev8 OST participants who participated in one 
or more high school planning activities are higher than those of non-participants. 

Figure I I. Number of types of high school planning activities engaged in by 8th graders in Elev8 schools during the 2010-1 I 
school year across all regions 


Elev8 OST Non-OST 



Number of High School Planning Activities Engaged In 


■ 0 ■ 1 ■ 2 ■ 3 ■ 4 ■ 5 ■ 6 ■ 7 


Similarly, as reported in Figure 12, 8 th graders who participated in Elev8 OST were more likely to have a 
plan to attend a competitive college-preparatory high school than other students. Specifically, 31% of 
Elev8 OST participants reported planning to attend a competitive college-preparatory high school, 
while just 17% of their non-participating peers reported similar plans. Additionally, students who 
participated in Elev8 OST were less likely to report planning to attend a neighborhood high school, or a 
public high school outside their neighborhood, than students in Elev8 schools who did not participate 
in these activities. 


s 6 Types, not numbers, of activities were recorded. A student who made ten school visits still only receives a count for one type of planning 
activity. We did not have access to a count of the number of times each student participated in a particular type of activity. 
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Figure 1 2: Types of high schools students plan to attend across regions in school year 2010-1 1 57 



A Competitive, College Preparatory High School 
Public High School NOT in Neighborhood 


Public High School in Neighborhood 
No Plan for High School 


Recognizing that each region is different, we also check if these differences in high school plans exist 
within each Elev8 region. Below, we see that high school planning distinctions are not consistent across 
regions. Figure 13 shows that in Region D, approximately equal percentages of Elev8 OST participants 
and non-participants reported planning to attend a competitive college-preparatory high school. In 
Region C a noticeable difference in high school plans between the two groups exists; however, it is 
smaller than in Regions A and B. Region A displayed the largest difference between the two groups’ 
high school plans. This rather drastic variation in high school plans between regions appears related to 
the different activities offered at the Elev8 sites, as well as the school choice policies and options in the 
regions. For instance, some schools in Region D are geographically isolated from any urban or 
suburban center and, therefore, school choice is unavailable. While students in other Elev8 schools in 
this region have some choice, they still have less opportunity for choice than students in the other three 
regions. 


57 Students could check all options that applied to them; therefore, the totals are greater than 100%. 


26 



Figure I 3: Types of high schools students plan to attend by region in the 2010-1 I school year 



o- 

Elev8 OST Non-OST 

Elev8 OST Non-OST 

Region C 

Region D 



Elev8 OST Non-OST Elev8 OST Non-OST 


| A Competitive, College Preparatory High School Public High School in Neighborhood 

Public High School NOT in Neighborhood | No Plan for High School 


In order to account for observed differences in student demographic characteristics and geographical 
region between the two groups, we conducted a series of analyses that allow us to estimate the 
magnitude of the relationship between Elev8 OST participation and high school plans and planning 
activities in isolation. In other words, we ask: Do Elev8 OST participants still report engaging in more 
planning activities and more frequently report planning to attend a competitive college preparatory 
high school if other contributing factors (i.e., SES, gender, r ace/ ethnicity, geographical region, years 
enrolled in a school) are the same? After taking into account pre-existing differences, there is no 
consistent evidence showing that Elev8 OST participants are more likely to have a high school plan, 
plan to apply for a competitive college preparatory high school, or attend more high school preparation 
activities than those students who did not participate in Elev8 OST. 

Given that OST participants and non-participants might be different in ways that we cannot account for 
in our data, and considering the wide variation in the dosage of Elev8 OST among participants, our next 
series of analyses examined whether and how participation levels in Elev8 OST related to OST 
participants’ high school planning, using data from Elev8 OST participants only. The results of our 
analyses suggest that greater frequency of participation in Elev8 OST is positively related to both the 
number of high school planning activities and the types of high schools that students plan to attend, 
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controlling for background characteristics that may be related to a student’s OST participation level. F * s 
More specifically: 

• Elev8 OST participants who attended a greater number of days were more likely to participate in 
additional high school planning activities than those who participated fewer days in the Elev8 
OST program. For example, a student who attended 43 days, the average number of days a 
participant attended Elev8 OST, was about 37 percent more likely to engage in one additional 
planning activity than a student who attended only 1 day of Elev8 OST programming. 

• Engaging in more high school planning activities is associated with a greater likelihood of 
planning to attend a competitive high school. Specifically, for each additional planning activity 
in which an Elev8 OST participant engages, he or she is 1.3 times more likely to report planning 
to attend a competitive college-preparatory high school. 

• Elev8 participants who attended more days of Elev8 OST programming are more likely to report 
planning to apply for a competitive college-preparatory high school than students who 
participated fewer days. For example, a student who attended Elev8 OST for 43 days was about 
1.4 times more likely to report planning to attend a competitive college-preparatory high school 
than a student who attended for only one day. 59 

While these findings do not definitely demonstrate that Elev8 OST participation is beneficial to 
students’ high school planning efforts beyond the benefit Elev8 may exert school-wide (irrespective of 
8 th graders’ participation in Elev8 OST), they do suggest that increased participation in Elev8 OST is 
associated with better high school planning efforts and higher high school aspirations within the 
subgroup of Elev8 OST participants. (Additional information on this analysis is presented in Appendix 
E). 

Summary 

Our analyses examine the relationship between student participation in Elev8 OST programs and 
outcomes in student school experience and high school transition planning. The data reveal that, among 
those who did participate in OST programs, higher levels of participation are associated with more high 
school planning activities. Further, students with higher participation levels are also more likely to have 
a plan to attend a competitive college-preparatory high school than those who participated fewer days 
in the OST programs. Because of differences in the Elev8 regions’ local school choice policies and 
opportunities, we deliberately left the structure of our question open to multiple interpretations. It is 
possible that in some localities charter schools would not be classified as competitive college- 
preparatory high schools, and instead would be considered a public school outside their neighborhood. 
Nonetheless, across the spectrum we see that students who participated more days in Elev8 OST are 
more likely to take advantage of school choice where it exists, and fewer planned to attend 
neighborhood public high schools, which many believe are lower performing than the competitive 
college-preparatory high schools. More research is necessary to determine if Elev8 OST students indeed 
attend higher-quality high schools, if they succeed in them, and if they are more likely to complete 9 th 
grade successfully. 

Notably, Elev8 is a whole-school model, and it is quite possible that it benefits all students who 
attended Elev8 schools— OST participants and non-participants alike. Our analysis suggests that both 


s 8 All findings reported are significant at ps.05. 

59 Again, while this assumption is verifiable in theory, we do not have reliable data on the names of the schools where students are planning to 
apply to high school, so we are unable to assess their quality. 
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Elev8 OST participants and non-participants reported high levels of efficacy, liked schools they 
attended, and valued school highly, and that a majority of students reported that they feel school is 
relevant to their future. Our analysis also suggests that the majority of 8 th -grade students in Elev8 
schools participated in high school planning activities and reported having a defined high school plan. 


Chapter 6: Final Thoughts 

Elev8 was created to address the myriad of challenges middle school students in chronically 
underserved communities face when striving to meet their educational potential. Across all sites, the 
initiative provides needed services tailored to the students, their families, and communities. These 
services include OST programming, family support services, and school-based health services. This 
report focuses on one component of Elev8: OST programming. It seeks to add to the literature by 
learning more about OST participation in a community school setting. 

Overall, Elev8 OST serves primarily low-income middle school students of color. However, the students 
who do participate in Elev8 OST are somewhat different from those students in Elev8 schools who don’t 
participate in Elev8 OST. Specifically, in some regions students who participate in Elev8 OST are more 
likely to be in a younger grade and of different racial/ ethnic background than students who do not 
participate. 

Measuring participation in OST is complex— and this complexity is exacerbated by the community 
school context where all students are exposed to other Elev8 activities in their schools daily. While the 
literature suggests there are multiple dimensions of OST participation, we were only able to investigate 
a few. We found that across all sites, forty percent of students in Elev8 schools participated in Elev8 
OST and those that did participated in Elev8 OST for an average of 43 days in the 2010-11 school year. 
This average compares favorably with the experiences of middle school students enrolled in programs 
similar to Elev8 OST. 60 The length of participation during the school year was robust. Students, on 
average, participated in Elev8 OST for over five months of the school year, and some regions achieved 
even longer average durations of participation. Finally, though the data available to us were limited, 
where data were available we saw that a majority of Elev8 OST participants were taking advantage of 
multiple components of the Elev8 initiative. 

While it has not yet been established that Elev8 improves student and school performance at this stage 
of the initiative, we sought to understand how participation in Elev8 OST relates to student outcomes. 
Though our study does not determine definitively if participation in OST is beneficial to students’ 
performance in school, we did learn that, on average, students who were enrolled in Elev8 schools 
(including both OST participants and non-participants) reported high levels of efficacy, liked schools 
they attended, and valued school highly. Further, a majority of the 8 th -grade students in Elev8 schools 
(including OST participants and non-participants) participated in high school planning activities and 
reported having clear high school plans. Notably, 8 th graders who participated in Elev8 OST more 
frequently were likely to have engaged in more high school preparation activities, and were more likely 
to report planning to take advantage of school choice and attend competitive college preparatory high 
schools than students who participated for fewer days in Elev8 OST. This is not explained by 
demographic differences between high-frequency and low-frequency participants. 


60 See Kauh, 2011. 
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Despite limitations, these findings suggest that increasing participation levels in OST programs in 
community school settings may promote participant success in high school. While we did not look at 
school-level changes in culture or consider other community school influences on the entire school 
population, findings suggest that there may be additional discrete benefits of participation in OST 
activities in this setting. As Elev8 schools strive to meet their central goal of preparing students for high 
school, OST activities may be a promising context to help inform students about their high school 
options and available planning activities. More research is needed to understand the relationship 
between Elev8 OST and positive student outcomes. 
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Appendix A: Data Sources 

Data used in this report come from the 2010-11 student survey and 2010-11 participation data. During 
this data collection period, all students in Elev8 schools were invited to participate in the annual Elev8 
survey. Additionally, all sites used a common database developed and managed by the evaluation team 
to track and record student participation. A summary of the data is provided in Table lA below. 

Table I A. The 2010-1 I sample of students included in the study 


Data 

Total 

Region A 

Region B 

Region C 

Region D 

Participation Data 

(participated at least one day in OST, 
with consents) 

2,476 

306 

1,073 

289 

808 

Survey Data 

(completed a survey, with consents) 

4,599 

465 

1,191 

1,260 

1,683 

Merged Sample (Participation + 
Survey Data) 

(linked OST student participation data 
with survey data; some individuals could 
not be linked) 

1,487 

238 

683 

135 

431 

Survey only sample 

(non-OST participants) 

2,988 

226 

505 

1,123 

1,134 


Elev8 student survey data 

The Elev8 survey was developed by the national evaluation team to understand the strengths, needs, 
and short-term outcomes of Elev8 students. The survey contains a combination of validated survey 
constructs along with additional questions of interest to Elev8 communities. Survey constructs and 
variables include: 

a) academic, health, demographic and stressor profiles of students; 

b) student involvement in OST and other positive activities; 

c) student barriers to participation in OST ; 

d) student utilization of school-based health centers and outcomes of that utilization; and, 

e) eighth graders’ plans and preparation activities for their transition to high school. 

Participation data 

Each site manages a Management Information System (MIS) with information on program participants 
including basic demographics and attendance in Elev8 services. Three Elev8 program components are 
captured in the MIS: extended-day learning activities, school-based health centers (aggregate), and 
family support programs (not linked to students). During the 2010-11 school year, the evaluation team 
managed the MIS for all regions, providing us with access to student-level identified data on a monthly 
basis. Consent forms from parents provide permission for the evaluation team to collect and analyze 
both participation and survey data. Students without permission are not included in the sample. 


31 




Students also provided assent to participate in the survey. Students without assent are not included in 
the sample. The data are administrative in nature, meaning that the demographics and attendance are 
reported by the program and not the participants themselves. 

Merged sample 

Data from the surveys were merged with data from the participation database to identify students who 
did and did not participate in Elev8 OST. The merge was conducted using two methods. First, the 
names of students in each database were linked by school and region. In cases where there was not a 
perfect match, researchers manually identified a match, if one existed. An imperfect match was one 
where a nickname was used (Joe for Joseph), a hyphenated name was used, there was a reversal in the 
order of the names (Eric Brown was listed as Brown Eric), or a middle name was indicated in one, but 
not both sources. 

In total, there are 4,599 students who consented and completed the survey, and 1,487 of them could be 
identified as OST participants (participated in the OST program at least one day) by linking them to 
participation data. In other words, among 4,599 students in survey file, 1,487 of them can be merged 
with participation data and thus identified as OST participants. The remaining 2,988 students who 
could not be linked to the participant file or could be linked but participated zero days in the program 
were considered non-OST participants. 
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Appendix B: Survey Methodology 

Data Collection Plan 


The survey methodology utilized in the spring of 2011 was designed to include all students in Elev8 
schools irrespective of their participation in Elev8 services. The goal of this methodology was to 
understand backgrounds and experiences of students in Elev8 schools. 61 The survey was also designed 
to help understand in-program and interim outcomes of students in Elev8 schools (several outcomes 
listed below). 

We worked with each school to distribute and collect consent forms so that as many students as 
possible were eligible to participate in the survey. Consent processes were aligned with the regulations 
of each school district. We gave each school six to eight weeks of lead time to distribute, collect, and 
submit consent forms to Public/Private Ventures (P/PV) staff. 62 Students who returned their consent 
forms, irrespective of whether permission was granted by their parents to complete the survey, were 
entered into a lottery for a prize. 

Instrumentation and Administration 

The instruments for this study were designed and developed by P/PV staff. Elev8 staff and local 
evaluators were provided the opportunity to review and provide suggestions on survey content. Many of 
the constructs used were from earlier P/PV work and many were developed from other scales used 
commonly in the field (see Appendix C for citations) including studies of the Boys and Girls Club and 
Big Brothers Big Sisters. Constructs for the survey included: 

• future orientation 

• school value 

• school liking 

• utilization of Elev8 services 

• stressors 

• high school preparation 

• sense of efficacy 

The survey days were scheduled in collaboration with Elev8 site staff, and generally took place in April 
or May 2011 on dates that were determined by the individual schools. Only students who had parent 
permission (consent) to participate in the survey were invited to take the survey and, of these students, 
only those who assented to participate took the survey. Students who did not have permission or did 
not assent were given an activity to work on or were instructed to work on their school work during the 
survey administration period. The survey was conducted in classrooms by trained survey 
administrators. Survey administrators read the survey aloud to students, although students were 
instructed to take it at their own pace if they liked. The survey administration process took 
approximately 30-45 minutes per classroom. Surveys were paper-based, as many of the schools do not 
have enough computers to deliver an online survey to multiple classrooms at one time. Once the 


61 Students were identified as out-of-school time participants or non-participants through linking their survey data with participation data 
collected through the Elev8 database. 

62 The national evaluation of Elev8 was originally conducted by Public/Private Ventures, a national non-profit research organization. 
Public/Private Ventures closed in July 2012, and the national evaluation of Elev8 was transferred by Atlantic Philanthropies to Research for 
Action, a Philadelphia-based non-profit organization working in educational research and reform to ensure opportunities and outcomes for all 
students. 
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surveys were completed, they were sealed by students, collected by survey administrators and mailed 
back to P/PV for processing and data analysis. 

Table I B: Spring 201 I survey response rates 


Region A 

Region B 

Region C 

Region D 

School 

Response 
Rate (%) 

School 

Response 
Rate (%) 

School 

Response 
Rate (%) 

School 

Response 
Rate (%) 

A 

83 

E 

60 

J 

59 

P 

76 

B 

90 

F 

49 

K 

68 

Q 

69 

C 

82 

G 

63 

L 

60 

R 

70 

D 

78 

H 

74 

M 

49 

S 

60 



1 

52 

N 

71 

T 

79 





0 

46 




Note: Response Rate = number of students in those grades who completed the survey -r number of students in 
school in Elev8 grades; students without consent and who did not assent are included in the denominator. 


Table 2B. Number of students completing the student survey (present, consented, and assented) 


Region A 

Region B 

Region C 

Region D 

School 

No. of 
students 

School 

No. of 
students 

School 

No. of 
students 

School 

No. of 
students 

A 

207 

E 

346 

J 

383 

P 

686 

B 

61 

F 

215 

K 

257 

Q 

456 

C 

123 

G 

286 

L 

249 

R 

145 

D 

74 

H 

291 

M 

83 

S 

282 



1 

53 

N 

135 

T 

114 





O 

153 



Total 

465 

Total 

1,191 

Total 

1,260 

Total 

1,683 
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Appendix C: Survey Measures 

School Liking (3 ITEMS: Scale from 1 (not very true) to 4 (very true); Source: Ilererra, 2004) 63 
I often feel excited at school 
I look forward to going to school every day 
I like school a lot 

School Efficacy (4 ITEMS: Scale from 1 (not very true) to 4 (very true); Source: Modified from 
Midgley et al., 2000) * * * 6 4 

I know I can do even the hardest work in my classes if I try 
I’m certain I can master the skills taught in school this year 
I can do almost all the work in school if I don’t give up 
I know even if my schoolwork is hard, I can learn it 

School Value (5 ITEMS: Scale from 1 (not very true) to 4 (very true); Original Source: Modified from 
Eccles et al., 1984 ) 6 s 

I feel that being a good student is important to me 
I would be upset if I got a low grade for one of my subjects 
I am interested in the things I learn at school 

I think school is useful in helping me make good decisions in my life 
I know what I learn in school is useful for the job I want as an adult 

Future Orientation/Connectedness (1 ITEM: Scale from 1 (not very true) to 4 (very true); Source: 
modified from Arbreton et al., 2009 66 ) 

I know doing well in school will help me in the future 


6 3 Hererra, C. (2004). School based mentoring: A closer look. Public/Private Ventures: Philadelphia. 

6 4 Midgley, C., Maehr, M.L., Hruda, L.Z., Anderman, E., Anderman, L., Freeman, K.E., Gheen, M...Urdan, T. (2000). Manual for the patterns 

of adaptive learning scales. The University of Michigan: Ann Arbor. MI. 

6 s Eccles, J.P., Adler, T. & Meece, J.L. (1984). Sex differences in achievement: A test of alternate theories. Journal of Personality and Social 
Psychology, 46, 26-43. 

66 Arbreton, A.J., with Bradshaw, M., Pepper, S. & Sheldon, J. (2009). Making every day count: Boys and Girls Clubs’ role in promoting 
positive outcomes for teens. Public/Private Ventures: Philadelphia. 
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Appendix D: Demographic Comparisons of Elev8 Participants with Non- 
Participants 

Table I D. Demographic comparisons of Elev8 OST participants and students who did not participate in Elev8 OST in 
school year 2010-1 I 


CHARACTERISTIC 

Region A 
n=464 

Region B 
n=l 188 

Region C 
n= 1 258 

Region D 
n= 1 564 


OST 

Non- 

OST 

OST 

Non- 

OST 

OST 

Non- 

OST 

OST 

Non- 

OST 

RACE/ETHNICITY 

African American 

1.7 

.4 

.9 

.6 

12.6 

18.8 

3.6 

3.8 

Asian 

86.9 

84.0 

42.6 

19.4 

21.5 

16.4 

4.5 

3.2 

Hispanic 

1.7 

1.8 

49.1 

76.6 

57.8 

57.1 

62.7 

69.7 

Native American 

6.4 

5.8 

3.7 

1.0 

.7 

2.4 

16.6 

8.9 

Other 

2.1 

5.3 

2.2 

1.0 

5.9 

4.3 

1.7 

1.3 

White 

1.3 

2.7 

1.5 

1.4 

1.5 

1.0 

1 l.l 

13.1 

GENDER 

Female 

55.7 

52.2 

52.3 

55.8 

40.0 

54.5 

58.5 

49.9 

Male 

44.3 

47.8 

47.7 

44.2 

60.0 

45.5 

41.5 

50.1 

FRL STATUS 

FRL 

86.4 

79.5 

72.3 

76.9 

64.9 

65.6 

67.1 

69.7 

NON-FRL 

10.2 

16.1 

8.3 

5.8 

6.0 

3.7 

13.6 

12.8 

Not Sure 

3.4 

4.5 

19.4 

17.3 

29.1 

30.7 

19.3 

17.5 

GRADE LEVEL 

5th Grade 

24.4 

21.7 

.2 

0.0 

0.0 

.4 

0.0 

.4 

6th Grade 

38.2 

32.7 

23.7 

15.5 

39.3 

29.6 

21.1 

15.8 

7th Grade 

22.7 

20.4 

41.9 

41.1 

40.7 

39.0 

37.1 

43.4 

8th Grade 

14.7 

25.2 

34.3 

43.6 

20.0 

31.1 

41.8 

40.3 
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Appendix E: Regression Analysis 

In order to examine the relationship between Elev8 OST participation and high school preparation 
activities and plans, we conducted two series of analyses. 

First, we conducted a series of logistic regressions to examine the potential differences between Elev8 
OST participants and non-participants in high school preparation activities and plans. The variables 
controlled in the statistical model include student gender, socioeconomic status, race/ethnicity, 
geographical region, and years in attendance at the school. The results did not provide consistent 
evidence that Elev8 OST participants are more likely to have a high school plan, plan to apply for a 
competitive college preparatory high school, or attend more high school preparation activities than 
those students who did not participate in Elev8 OST. Our analyses may be biased because we are unable 
to account for all the differences between students who did and did not participate in Elev8 OST 
activities. Further, students were not randomly selected to participate in the Elev8 OST program; 
instead, they chose to, or were encouraged to, participate. Additionally, the estimate may be biased 
because some students may have received other, non-Elev8, OST or enrichment programs that are not 
reported in our data. 

Second, using participation data, we explored how the frequency of participation in Elev8 OST was 
associated with high school preparation activities, and, in turn, how planning activities were associated 
with where the student planned to attend high school, controlling for observed student and school 
characteristics, as described above. Figures lE through 3E display some of the findings from these 
analyses. In each figure, solid lines represent the change in estimated probabilities, the dashed lines 
represent the 95% confidence intervals, and the vertical lines (x=43) display the average days students 
participated in Elev8 OST program. All the correlations displayed in the figures are significant at 
p<0.05 level. 
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Figure I E: Students who attended more days in Elev8 OST were more likely to participate in high school preparation 
activities 



n = 474 sth Grade students Days Attended in Elev8 OST Program 


Figure 2E: Engaging in more high school planning activities is associated with a greater likelihood of high school planning 



Number of High School Planning Activities 
Having a High School Plan 

Having a Plan to Attend Competitive College Preparatory High Schools 


n = 474 8th Grade Students 
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Figure 3E: Students who attended more days of Elev8 OST are more likely to report planning to apply for a competitive 
college-preparatory high school 
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